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Since  the  early  1960s,  the  federal  government  has  been  involved  in  the 
direct  creation  of  new  employment  opportunities,  especially  at  the1  state 
and  local  level.   Commerce  Department  programs  provide  intergovernmental 
grants,  business  loans,  planning  assistance  and  funds  for  public  works 
("infrastructure"),  all  aimed  toward  stimulating  private  investment  in 
new  (or  retooled)  productive  capacity.   The  Community  Services  Admini- 
stration (formerly  0E0)  allocates  equity  capital  to  community  development 
corporations  to  facilitate  the  creation  or  retention  of  locally-owned 
and  managed  "quasi-public"  enterprises  in  poverty  areas.   And  since  1971, 
the  Department  of  Labor  has  operated  a  series  of  large  (and  growing) 
programs  to  finance  public  service  employment  in  state  and  local  govern- 
ments.  Other,  analogous  programs  are  currently  on  the  drawing  boards  of 
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EDA  are  responsible  for  the  opinions  or  arguments  expressed  herein.   We 
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ceptual issues.  ~. 
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Washington  planners,  and  now  a  number  of  state  governments  have  begun  to 
undertake  job-creation  experiments  of  their  own.—   Given  the  prevailing 
forecasts  of  continued  high  unemployment  for  the  American  economy  — 
especially  for  its  older  urban  and  rural  areas  —  there  can  be  no  doubt 
that  these  efforts  to  supplement  aggregate  fiscal  and  monetary  policy 
with  programs  to  "target"  job-creation  to  disadvantaged  workers  and  places 
will  grow  in  the  years  to  come. 

In  the  evaluation  of  any  such  program, one  especially  thorny  methodolo- 
gical problem  presents  itself.   How  are  we  to  decide  what  would  have  happened 
to  the  workers  who  actually  get  the  newly-created  jobs,  in  the  absence  of  the 
program?   Suppose  (say)  a  public  works  or  public  service  employment  program 
provides  a  budget  for  100  weeks  of  full-time  wages,  salaries  and  overhead. 
Does  this  mean  that  100  weeks-worth  of  new  work  has  been  created?  Probably 
not.   From  the  point  of  view  of  both  the  individual  enrolled  in  the  position 
and  the  program  managers,  the  net  change  in  employment  (expressed  in  weeks) 
is  equal  to  the  difference  between  100  and  the  number  of  weeks  that  the  enrol- 
lee  could  have  expected  to  work  if  the  job-creation  program  had  never  existed. 

Since  most  workers  face  the  prospect  of  at  least  some  employment  over  the 

2/ 
course  of  a  year,—  the  net  increment  to  their  work  experience  probably  is 

less  than  100  (weeks) . 


—  Cf.  Audrey  Prager,  et  al. ,  Job-Creation  in  the  Community:   The  Massachu- 
setts Local  Initiatives  Program  (Cambridge:   Abt  Associates,  1977). 

2/ 

—  In  1975,  only  15%  of  those  with  some  unemployment  during  the  year  were 

without  work  the  entire  time.   US  Dept.  of  Labor,  Employment  and  Training 
Report  of  the  President  (Washington,  DC:   Government  Printing  Office,  1977), 
p.  216. 
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We  could,  of  course,  simply  assume  either  that  the  program  creates  no 
additional  employment  for  the  individuals  involved  (everyone  enrolled  would 
have  worked  for  the  same  total  number  of  weeks  over  the  program  period)  or 
that  all  the  employment  is  additional  (none  of  the  participants  would  have 
found  any  work  without  the  program) .   This  latter  assumption  might  conceivably 
hold  for  those  who  were  either  structurally  unemployed  or  who  were  brought  back 
into  the  labor  force  as  a  result  of  the  program.   The  former  might  hold  true 
for  participants  who  were  either  employed  prior  to  the  program  or  were  unem- 
ployed for  a  very  short  (frictional)  time  before  their  entry  into  the  program. 
The  truth,  of  course,  lies  somewhere  between  the  extremes. 

The  ideal  solution  to  this  "unobserved  variable"  problem  in  program 
evaluation  is  to  draw  an  independent  control  group  of  non-program  participants 
at  each  project  site,  matched  to  a  sample  of  program  enrollees  by  age,  race, 
sex,  prior  labor  market  experience,  etc.  and  then  to  compare  their  experiences 
with  participants  during  the  program  period.   Unfortunately,  as  is  well  known 
to  program  evaluators,  control  groups  are  expensive  to  find,  feed  and  follow. 

And  even  the  absolute  technical  superiority  of  this  method  has  been  called 

3/ 
into  question  by  some  evaluators.— 

Our  solution  to  the  problem  has  been  to  make  use  of  recent  developments 

in  the  quantitative  modeling  of  the  process  by  which  workers  move  in  and  out 


3/ 

—  Cf.  "Three  Perspectives  on  Social  Science  Research",  Monthly  Labor  Review, 

Feb.  1972,  on  cross-section  sample  surveys,  longitudinal  surveys,  and 

experiments  with  control  groups. 
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4/ 
of  the  various  "states"  in  the  labor  market.—   Using  an  econometric  model 

of  employment  transitions  for  various  age-race-sex  cohorts  of  the  national 
population,  we  simulate  the  expected  work  experiences  of  each  cohort  over 
time.   From  these  simulations,  we  are  able  to  extract  estimates  of  the 
expected  amount  of  time  at  work  for  each  cohort.   These  statistically  simu- 
lated weeks  of  work  are  then  used  to  discount  the  actual  volume  of  work 
generated  by  the  Job  Opportunities  Program  of  the  Economic  Development 
Administration,  a  countercyclical  job-creation  program  that  was  in  effect 
between  June  1975  and  July  1977.   The  "Title  X"  Program  (so-called  because 
it  was  passed  into  law  in  1975  as  a  new  Title  X  to  the  Public  Works  and 
Economic  Development  Act  of  1965)  directly  funded  over  75,000  person-years 
of  work  and  is  quite  typical  of  the  kind  of  short-term  targeted  job-creation 
program  which  we  are  likely  to  see  more  of  in  the  years  ahead. 


Project  Evaluation  in  a  Federal  System: 
The  Interdependence  of 
Efficiency  and  Equity  Considerations 


Before  proceeding  to  a  description  of  the  model  and  how  it  was  used  to 
assist  us  in  evaluating  the  EDA's  Title  X  program,  we  want  to  place 
this  particular  methodological  problem  into  a  larger  context.   There  has  been 
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—  There  has  been  much  research  into  these  issues,  especially  in  reference 

to  national  employment  policy.   See,  for  example,  George  L.  Perry,  "Unemploy- 
ment Flows  in  the  US  Labor  Market",  Brookings  Papers  on  Economic  Activity, 
Vol.  2,  1972;  Stephen  Marston,  "Employment  Instability  and  High  Unemployment 
Rates",  BPEA,  Vol.  1,  1976;  Robert  L.  Crosslin  and  David  W.  Stevens,  "The 
Asking  Wage-Duration  of  Unemployment  Relation  Revisited",  Southern  Economic 
Journal,  Vol.  43,  No.  3,  January  1977;  Charles  Holt,  "Modeling  a  Segmented 
Labor  Market",  in  Phyllis  A.  Wallace  and  Annette  M.  La  Mond ,  eds.,  Women , 
Minorities,  and  Employment  Discrimination  (Lexington,  Mass:   D.C.  Heath, 
1977). 
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growing  discussion  of  late  about  the  many  ways  in  which  local  implementation 
of  an  initially  federally  promulgated  program  can  undermine  the  intentions 
of  the  federal  planners.   We  are  bombarded  by  a  whole  new  set  of  technical 
terms  that  have  something  to  do  with  these  intergovernmental  relations : 
"budget  substitution  effects",  "displacement  effects",  "vacuum  effects". 
Our  objective  in  this  brief  section  is  to  sort  out  these  concepts  and  to 
show  how  their  interpretation  relates  to  questions  of  who  benefits  (and 
loses)  from  job-creation  policy  as  well  as  the  question  of  how  many  jobs  are 
"really"  created.   This  will  give  us  a  structure  within  which  to  pose  our 
own  particular  question:   how  much  of  the  labor  demand  created  by  a  govern- 
ment program  would  have  been  available  to  the  enrollees  anyway,  even  if 
the  program  had  never  existed? 

Consider  the  federally- funded  local  employment  project  designed  to  create 
100  person-weeks  of  new  labor  demand  (fig.  1).   Let  us  assume  that  the  local 
government  hires  new  enrollees  to  perform  50  weeks  of  work.   The  other  50 
weeks  of  labor  are  performed  by  workers  already  "on  board"  in  the  local 
government.   This  is  the  budget  substitution  effect;  in  fig.  1,  it  occurs 
at  a  rate  of  50/100  or  .5.   Moreover,  in  the  absence  of  the  federal  program, 
suppose  that  the  new  enrollees  could  have  found  10  weeks  of  work  anyway, 
somewhere  in  the  economy,  so  that  only  40  weeks  of  new  work  ("new"  to  the 
enrollees)  is  actually  created.   Now,  suppose  in  addition  that  the  local 
government  deliberately  chooses  not  to  use  20  of  the  weeks  for  which  it  has 
funding,  and  implements  this  decision  by  displacing  (i.e.  laying  off) 
people  who  would  have  performed  this  work  to  make  room  for  newly  hired 
enrollees  in  the  program.   This  is  a  displacement  effect  of  20/50  =  .4. 
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But  then  only  30  of  the  50  "new"  person-weeks  of  work  are  truly  new.   Moreover, 
only  40/50  of  those  weeks,  or  24  weeks,  provide  work  that  the  new  job- 
holders wouldn't  have  had  in  the  absence  of  the  program.   Note  that  the 
substitution  and  displacement  processes  free  up  70  person-weeks  worth  of 
resources  which  the  local  jurisdiction  can  use  for  a  variety  of  other  pur- 
poses, from  direct  expenditure  to  debt  service  or  even  tax  relief. 

It  seems  to  us  that  there  is  no  one  employment  impact  to  be  read  from 
this  scenario.   The  results  depend  crucially  on  the  viewpoint.   Suppose 
we  evaluate  the  outcome  from  the  perspective  of  the  program  managers. 
The  federal  objective  was  to  create  100  weeks  of  project  employment.   The 
local  government  is  actually  using  only  80  weeks,  having  chosen  (via  dis- 
placement) not  to  use  20  person-weeks  worth  of  resources.   But,  as  we 
have  just  seen,  not  all  of  these  80  weeks  constitute  new  work. 


net  impact 
to  program 
managers 


M  [.,..  .,..,J 

\impact/y_   \  substitution/] 


("weeks  of  worki 
I  expected  by 

employees  in 

absence  of 


program 


weeks  of 
—  displaced 
employment 


weeks  of  work 
funded  by 
program       I 
(i.e.  intended] 
_impact)      JJt 

Assume  that,  just  as  new  employees  could  have  found  a  fifth  of  their 
new  weeks  of  work  elsewhere   in  the  absence  of  the  program,  so  a  fifth  of 
the  displaced  person-weeks  can  be  made  up  elsewhere  in  the  economy; 
1/5  of  20  =  4. 

Applying  this  algorithm  to  the  example  in  figure  1,  net  impact  is 

new  weeks  of  job-creation. 

A  second  perspective  for  program  evaluation  is  that  of  the  new  employees 
themselves.   Since  displacement  does  not  directly  affect  them, 


weeks  of  work 
expected  by 
the  displaced 
employees 


applying  tnis  aigoritnm  to  the  example 

H  (>  -  M  -  iffl  - » ♦ » -  * 


100 
intended 


+30  A     new 


substitution 
of  50  from  some 
>ther  pre-existing 
payroll  to  project 
payroll 


(generates  50  weeks 
worth  of  resources 
available  for 
other  uses) 


new  displacing  old 


old 


old  displaced 

•s   (generates  20  weeks  worth 
N  Nof  resources  available 
for  other  uses) 


Figure  1 

Possible  Allocation  of  Federal  Resources 
Within  a  Local  Project 


*obtained  by  applying  the  proportion  40/50 
to  each  of  the  components  30  and  20. 
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Using  the  numbers  in  our  example,  new  employees  gafned 


sing  the  numbers  in  our  example,  t 


—  ]  I   =40  new  weeks  of  work. 
There  is  yet  a  third  viewpoint  for  evaluating  the  outcome:   that  of  the 
economy  as  a  whole  (the  raacroeconomic  or  "social"  perspective).   The  federal 
government  provided  100  person-weeks  worth  of  new  resources.   The  local 
government  directly  used  only  80  weeks  worth.   Moreover,  50  of  those  80 
weeks  of  labor  demand  were  filled  by  persons  already  on  board ,  reducing 
the  new  demand  to  30  weeks.   From  the  social  perspective,  the  full  30  is 
relevant,  rather  than  the  24.   If  we  assume  that  there  is  still  a  significant 
amount  of  involuntary  unemployment  in  the  economy  as  a  whole,  even  the  fact 
that  project  employees  could  have  found  6  weeks  worth  of  work  in  the  absence 
of  the  program  is  no  longer  relevant;  those  6  weeks  will  be  claimed  (or  the 
slots  will  become  occupied)  by  people  who  are  now  presently  unemployed  some- 
where: the  vacuum  effect.   Moreover,  the  social  (as  opposed  to  either  the 
managerial  or  private)  impact  is  much  greater  than  30,  because  70  weeks 
worth  of  job-creating  resources  have  been  freed  up  for  use  outside  the 
program.   As  Charles  Killingsworth  has  pointed  out,  there  is  no  reason  not 
to  assume  that  these  resources  will  not  ultimately  be  respent  somewhere  in 
the  economy.—   To  the  extent  that  they  are  respent  (rather  than  saved), 
and  that  the  new  uses  generate  an  equivalent  demand  for  labor,  the  net  social 
impact  will  approach  the  intended  impact  of  100  new  person-weeks  of  labor 
demand. 


-  Charles  C.  Killingsworth,  "The  Role  of  Public  Service  Employment", 
Industrial  Relations  Research  Association/Proceedings,  1977  Annual  Spring 

Meeting,  1977. 
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So  much  for  the  theoretical  possibilities.   Do  we  have  any  empirical 

evidence  on  these  parameters  of  sub-federal  fiscal  behavior?   Some  econometric 

research  has  been  conducted  on  the  substitution  effect,  indicating  rates  of 

anywhere  from  40%  to  90%  in  the  short  run.—   But  Killingsworth  quite 

rightly  says  of  these  estimates  that 

the  studies  on  which  (they)  relied  were 
based  on  speculation  to  such  an  extent 
that  they  could  not  properly  be  given  < 
any  weight  as  evidence. .. far  too  much 
has  been  made  of  the  "substitution" 
argument.   We  have  no  solid  evidence 
concerning  its  extent;  in  fact,  no  one 
has  even  thought  of  a  reliable  way  to 
measure  it.?. 

~~  g  / 

Wiseman  also  calls  the  prevailing  estimates  "seriously  flawed".—   As  far 

as  the  evidence  of  the  displacement  effect  is  concerned,  we  have  only  anec- 

97 

dotal  accounts  so  far.—   It  seems  to  us  that,  for  rather  obvious  political 

reasons  (displaced  persons  will  often  get  to  vote  their  dissatisfaction 
later),  this  effect  is  likely  to  be  small. — 

Thus  we  can  see  that  some  measure  of  a  program  enrollee's  expected  work 


—  National  Planning  Association,  An  Evaluation  of  the  Economic  Impact  of  the 
Public  Employment  Program  (Washington,  D.C. :   N.P.A.,  1974);  George  Johnson  and 
James  Tomola,  "The  Efficacy  of  Public  Service  Employment  Programs",  US  Dept. 

of  Labor,  Office  of  Policy  Evaluation  and  Research,  Technical  Analysis  Paper 
17A,  1975.   These  and  other  related  studies  are  critically  reviewed  by  Michael 
Wiseman,  "Public  Employment  as  Fiscal  Policy",  Brookings  Papers  on  Economic 
Activity,  Vol.  1,  1976. 

—  Killingsworth,  op.cit. 

8/ 

—  Wiseman,  op.cit. ,  who  also  offers  the  observation  that  the  problem  itself 

seems  correctable  by  appropriate  administrative  sanctions  or  some  sort  of 
performance  contracting  system. 

9/ 

—  "Chicago  to  Repay  Title  VI  Funds  Misused  in  City's  Jobs  Program",  ETA 

Interchange,  US  Dept.  of  Labor,  October  1977,  p. 2. 

—  For  completeness,  we  may  mention  another  factor  which  has  to  be  considered 
by  program  evaluators.   The  creation  of  jobs  or  —  more  often  —  training  slots 

intended  for  some  target  group  may  attract  new  entrants  into  the  labor  force, 
some  of  whom  may  actually  get  the  new  jobs.   Whether  or  not  this  affects  the 
private  or  social  benefits  of  the  program  in  terms  of  actual  new  work  created, 
it  will  prevent  the  unemployment  rate  from  declining  as  much  as  would  have 
occured  in  the  presence  of  an  inelastic  supply  of  labor.   Thus,  the  value  of 
the  official  unemployment  rate  as  an  indicator  of  program  impact  is  compromised 
when  labor  supply  is  variable. 
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experience  in  the  absence  of  that  program  is  an  important  parameter  in 
evaluating  the  net  benefits  to  the  enrollee  him(or  her) self,  as  well  as 
to  those  officials  in  charge  of  the  program  (it  is  not,  however,  germane  to 
the  estimation  of  net  social  benefits).   We  now  proceed  to  an  explanation 
of  how  we  estimated  that  parameter  in  connection  with  our  evaluation  of 
the  Job  Opportunities  Program  of  EDA. 

The  Labor  Market  Transition  Model 

In  the  course  of  our  research,  we  became  aware  of  the  job-search-theory- 
based  work  at  the  Urban  Institute,  specifically  the  "Inflation  and  Unemploy- 
ment Project"  directed  by  Dr.  Charles  Holt.   With  the  enthusiastic  support 
and  advice  of  Drs.  Holt,  Ralph  Smith  and  Jean  Vanski,  we  proceeded  to  adapt 
the  model  constructed  at  the  Institute  to  meet  our  own  program  evaluation 
needs. 

Smith,  Vanski  and  Holt  had  collected  month-to-month  "gross  flows"  data 

from  the  Current  Population  Survey,  covering  the  period  between  July  1967 

and  December  1973  (5.3  years  x  12  months/year  =  66  months  of  data),  a  period 

including  one  complete  business  cycle.   For  each  of  sixteen  demographic 

groups: 

male,  white,  aged  16-19 

20-24 


II 

25-59 

II 

60+ 

male,  nonwhite, 

aged 

16-19 

ti 

20-24 

it 

25-59 

it 

60+ 

female,  white, 

aged 

16-19 

ii 

20-24 

M 

25-59 

ii 

60+ 
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female,  nonwhite,  aged  16-19 

20-24 
25-59 
60+ 


ii 
ii 


Smith,  et  al. ,  constructed  a  string  of  month-to-month  employment  status 
transition  matrices,  with  the  following  structure: 


Employed  last  month 

Unemployed  last  month 

Not  in  the  labor 
force  last  month 


Employed 
this  month 


'11 


'21 


'31 


Unemployed 
this  month 


'12 


'22 


'32 


Not  in  the 
labor  force 
this  month 

P13 
p23 

P33 


Each  element  (p..)  represents  the  probability  that  a  person  of  the  corre- 
sponding age/race/sex  will  find  him(her)self  in  a  particular  status  this 
month,  given  his (her)  status  last  month.   The  probabilities  sum  to  1.0  (or 
a  hundred  percent} along  the  rows  ,  Thus,  for  example,  of  everyone  employed 
last  month,  p   were  employed  this  month,  p..  „  were  unemployed,  and  p   dropped 
out  of  the  labor  force;  p...  +  p._  +  p   =  1.0.   For  each  demographic  group, 
we  therefore  have  a  string  of  (66-1=) 65  transition  matrices  which  describe 
how  people  of  given  age/race/sex  move  in  and  out  of  jobs  and  in  and  out  of 
the  labor  force  from  month  to  month. 

Smith  et  al.  then  employ  multivariate  regression  analysis  to  measure 
the  correlates  of  these  month-to-month  variations  in  the  p's. —   Simplifying 


—  Ralph  E.  Smith,  "A  Simulation  Model  of  the  Demographic  Composition  of 
Employment,  Unemployment,  and  Labor  Force  Participation",  in  Ronald  G. 
Ehrenberg,  ed.,  Research  in  Labor  Economics,  Vol.  I  (Greenwich,  Conn: 
JAI  Press,  1977). 
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their  presentation  somewhat,  we  have  the  equivalent  of  (9x16=) 144  equations, 

one  for  each  type  of  transition  for  each  demographic  group: 

p , .   =  f  (time  trend;  seasonal  adjustment  factors,  ratio  of  the 
Conference  Board's  estimated  level  of  job  vacancies 
to  the  Current  Population  Survey  unemployment  level 
lagged  one  month,  called  V/U) 

where  d  =  1,2,. ..,16  demographic  groups 

i  =  1,2,3  initial  "states" 

(last  month's  status) 

j  =  1,2,3  current  states 

(this  month's  status) 

and  there  are  t  ■  1,2,..., 65  month-to-month  changes  being  observed.   The 

ratio  (V/U)  in  month  (t-1)  is  an  important  variable  measuring  the  tightness 

of  the  labor  market,  i.e.  "the  availability  of  jobs  in  relation  to  the 

12/ 
availability  of  people  to  fill  them". — 

From  actual  program  data,  we  know  the  age,  race,  sex,  program  entry  and 

exit  months,  and  month  of  enrollment  of  each  Title  X  Job  Opportunities 

Program  enrollee.   From  CPS  and  Conference  Board  data,  we  can  measure 

(V/U)  _. ,  the  labor  market  conditions  in  each  of  the  enrollee 's  months  in 

the  program  (lagged  one  month) .   By  substituting  the  appropriate  Title  X 

project  period  data  into  the  Urban  Institute  equations,  we  can  forecast  the 

expected  pattern  (time  path)  of  transition  probabilities  for  each  Title  X 

employee.   Appropriate  cumulation  of  the  forecasted  conditional  probabili- 


12/ 

—  Ibid.   Smith  notes  that 

"The  US  does  not  have  comprehensive  vacancy  statistics.   We 

assume  an  average  vacancy  level  during  the  1967-73  estimation 

period  of  about  2  million  and  use  the  Conference  Board's  Index 

to  measure  the  variation  (from  month  to  month)  in  the  level." 
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13/ 
ties  of  being  employed  each  month, —  counted  over  the  range  of  months  in 

the  program,  then  yields  the  needed  estimate  of  the  unobserved  variable: 
the  number  of  weeks  that  the  enrollee  would  have  worked  in  the  absence  of 
EDA's  Job  Opportunities  Program. 

First,  however,  we  must  realize  that  the  Smith  et  al.  model  is  not  quite 
so   straightforward  as  we  have  presented  it  above.   In  particular,  it  is 
aggregated  into  cohorts  (i.e.,  it  does  not  literally  forecast  the  mobility 
patterns  of  individuals),  and  the  population  of  each  cohort  varies  from 
month  to  month,  changing  the  size  of  the  pool  of  people  entering  the  system 
being  modeled  (in  other  words,  the  system  is  not  closed).   Operationally, 
this  means  that  the  transition  process  is  not  strictly  Markovian,  so  that 
nine  transition  probabilities  p   cannot  in  fact  be  consistently  estimated 
by  nine  equations. 

The  Smith  et  al.  solution  is  to  estimate  the  nine  transition  probabili- 
ties (for  each  of  the  sixteen  demographic  groups)  with  six  equations  and 
five  identities  that  allow  for  an  intermediate  state  called  "labor  force 
(re)entry"   (which  may  or  may  not  be  successful  in  terms  of  whether  or  not 
it  results  in  finding  a  job)  and  which  account  for  month-to-month  variations 
in  the  size  of  the  population  in  the  system.   The  six  behavioral  equations 
which  —  together  with  two  identities  —  permit  estimation  of  the  off-diagonal 
probabilities  are  of  the  form! 


13/ 

—  The  algorithm  for  this  cumulation,  reproduced  below,  assumes  that  transi- 
tion from  one  month  to  the  next  is  a  first-order  Markovian  process,  i.e.  that 
where  you  end  up  next  month  depends  only  on  where  you  were  the  month  before, 
and  not  on  how  you  got  to  that  point.   This  is  a  simplification  of  the  transi- 
tion process,  no  doubt,  but  it  is  the  only  assumption  consistent  with  the 
first-order  difference  equation  form  of  the  Urban  Institute  model. 


P.     -  a(M6 


YT     t=l 
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e.g.,  for  the  unemployment-to-employment  transition, 

/    \  n 

lne  (^  lne  «  +  *  ta.©^  +  YT  +  iJ  6t  St 

where  the  S's  are  seasonal  dummies,  indexing  the  month,  and  T  is  an  annual  time 
trend  indexing  the  year.   There  are  three  identities  for  the  diagonal  proba- 
bilities, constructed  to  allow  for  changes  in  the  various  stocks  in  the  model 
(population,  total  labor  force,  etc.). 

Smith  et  al.  run  their  estimated  model  by  inputting  initial  stock  values, 
simulating  one  month's  flows,  and  then  observing  the  new  stock  values.   These 
endogenously  generated  stocks  then  become  input  into  the  next  month's  simulation 
and  so  on.    Since  we  have  exogenous  vacancy  and  unemployment  data  for  each 
month  of  our  simulation  period  (1975-77),  from  Conference  Board  and  CPS 
records,  we  chose  to  use  the  Urban  Institute  model  to  compute  wholly  self- 
contained  month-by-month  forecasts.   That  is,  instead  of  simulating  new 
stocks  each  month,  we  entered  them  anew,  from  outside  the  model.   The 
mechanics  were  simplified  in  this  way,  but  our  forecasts  became  inconsistent, 
occasionally  producing  transition  matrices  whose  row  sums  exceeded  unity.   To 
adjust  for  this,  we  decided  to  use  only  the  six  behavioral  equations  and  two 
identities  that  generate  the  off-diagonal  probabilities,  and  to  constrain  the 
diagonal  probabilities  to  values  that  would  preserve  the  condition  that  row 
elements  sum  to  unity.   There  is  some  loss  of  Information  here,  because  the 
six  behavioral  equations  do  not  take  into  account  the  levels  of  stocks  of 
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employed  persons,  unemployed  persons,  etc.   But  since  our  subsequent  simu- 
lation of  cumulative  weeks  worked  (see  below)  requires  that  the  strings  of 
forecasted  transition  matrices  be  consistent  with  a  first-order  Markovian 
transition  process  (i.e.,  row  elements  sum  to  unity),  we  really  had  no 
choice. —   The  matrices  for  a  sample  of  cohorts  are  given  in  Tables  1-4. — 
Inspection  reveals  realistic  patterns,  especially  in  terms  of  seasonality 
(e.g.,  young  workers  show  the  most  volatile  changes  in  labor  force  entry 
and  exit  at  the  beginning  and  end  of  summer).   It  remains  now  to  explain 
how  we  use  these  forecasted  transition  tables  to  estimate  "weeks  worked  in 
the  absence  of  Title  X". 


14/ 

—  It  is  hard  to  pin  down  the  cause  of  our  inability  to  produce  consistent 

forecasts  with  the  Urban  Institute  model  in  its  original  form,  because  the 
behavioral  equations  in  that  model  were  estimated  by  Ordinary  Least  Squares. 
Since  the  dependent  variables  are  probabilities  bounded  by  0  and  1,  OLS 
produces  estimates  which  are  heteroscedastic,  and  which  do  not  rule  out 
forecasts  that  fall  outisde  the  0-1  interval.   The  correct  estimation  tech- 
nique is  probit  analysis  (since  those  left-hand  variables,  at  the  Smith 
et  al.  level  of  aggregation,  are  ratios).   We  considered  re-estimating  the 
basic  equations  themselves,  but  the  cost  would  have  been  prohibitive.   The 
upshot  is  that  forecasts  of  transition  probabilities  that  do  not  sum  to  unity 
across  rows  are  quite  possible  with  the  system  that  the  Urban  Institute 
has  generated. 

—  The  complete  set  of  16  strings  of  transition  matrices  is  available  in 
Abt  Associates,  Effects  of  Job  Creation,  Volume  Four,  Appendix  B,  Cambridge, 
1977.   The  format  of  these  tables  corresponds  to  the  matrix  described  on 

p.  11  above. 
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Simulation  of  Expected  Weeks  of  Work  for  a 
Sample  of  Job-Creation  Program  Enrollees 


We  have  records  on  a  stratified  cluster  sample  of  2,000  Title  X  workers. 
Each  is  characterized  by  age  (at  time  of  program  entry),  race  and  sex, 
employment  status  in  the  month  prior  to  program  entry,  and  the  months  of 
program  entry. and  exit. 

The  person's  age/race/sex  tell  us  which  of  the  16  transition  matrix  strings 
to  consult.   The  entry  and  exit  months  tell  us  where  to  enter  and  from  which 
point  to  leave  the  string.   The  pre-enrollment  status  tells  us  through  which 
row  (or  "window")  of  the  entry  month  matrix  we  are  to  enter  (row  1  =  "employed", 
row  2  =  "unemployed",  row  3  =  "not  in  the  labor  force").   We  then  "walk"  the 
person  through  the  subsystem  of  transition  probabilities  so  selected,  compute 
the  cumulative  number  of  expected  months  spent  in  the  "employed"  state,  and 
multiply  that  number  by  4.3  (average  weeks  in  a  month). 

For  each  cohort,  it  is  possible  to  compute  a  table  of  expected  weeks  of 
employment  for  all  possible  entry/exit  month  pairs,  for  each  of  the  three 
possible  pre-enrollment  statuses.   For  each  Title  X  worker,  we  can  then 
consult  the  appropriate  pre-computed  table,  as  an  alternative  to  actually 
computing  the  month-by-month  expectations  for  each  of  the  2,000  workers  in 
our  sample.   The  Fortran  extract  we  wrote  to  simulate  these  expectations  is 
reproduced  as  Figure  2, —  followed  by  a  sample  of  output  tables  for  four 
of  the  16  cohorts  (tables  5-8)  [for  a  complete  set  of  results,  refer  to  fn. 
15].   Referring  to  Table  5  by  way  of  example,  we  estimate  that  a  previously 


—  In  the  program  extract,  TP(I,J,K)  is  a  transition  matrix  for  month  K, 
whose  rows  (I  =  1,2,3)  represent  employment  status  at  the  beginning  of  the 
month,  and  whose  columns  (J  =  1,2,3)  represent  the  status  at  the  end  of  the 
month. 
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Fortran  Simulation 
Algorithm 
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DO    4H1     11=1,3      

DO    123    JJ=2,?2 
c DO    123    KK=JJ,22 

C  COMPUTE    ESTIMATED    KEEKS    OF    EMPLOYMENT 

C  KK    =    EXIT    MONTH    (NOV     1975=2) 

C 

*"..'.      ' '  jji  =  jj*i 

KKl     =    KK-1  * ' 

RANGE=1+(KK"JJ) 
FACTOR=RANGE 
--™-:.";:IF(  RANGE.  GT.URANGE=4  •-- 

DO    102    1  =  1,3  --~      -     .. 

DO    102    J=l,3 
W(I,J)  =  TP(  I,J, JJl) 

•■«  --'   .tioci,  j)=o.o  ■•■" -~ - — ...- 

A(I)=0.0  "    -'"—-*"——- ...      .    .     • 

102         C  ONTI  HUE 

„ DO     100  2    1=1, 4 

.      WEEKS£I)  =  0.0       ""' —    -•■•■       -- -•- 

1002       CONTINUE         ~    """"" ~  — — .„.„ 

TWKS=0.0 

.      WKS=C.O 

C   """-"END  OF  INITIALIZATIONS  FOP  EKKS  COMPUTATIONS 

-  c      G0T0  f  HO,  111,  112,  113),  RANGE 
C       MODULE  FC?  1  MONTH  IN  PROGRAM 

•  c   ..iiUTU  100     .  __  -  ■  - —,.;-.- 

C      MODULE  FOR  2  MONTHS  IN  PROGRAM 

1111    DO  1111  1=1,3  ■  ~   

TWKS  =  TWKS  +  TP(II#I,'JJ)*ip/i  i  kk  »  - 

1111   CONTINUE        ,rm'i*JJi*TP(I,l,KK) 

WEEKSCRANGE>=TKKS 

-  GOTO  100  •  • '■■■>  ~  ■ 
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C  HQDULE    FOR     3     MONTHS     IN    PROGRAM 

.112    .     00    111?    1  =  1,3 
00    111?    JM-3 

1112       CGNTl56c>'TPCII'J'JJ>*TP(J't'JJ1> 

...        03    1113    1=1*3 

ins    cT8iisnf,liiP*u,*n'n'I'KI<' 

WEEKS<*ANGF)=TWKS 

.«.  GOTO  100 

C 

C      MODULC  FCR  4  MONTHS  OP  MGGE  IN  PROGRAM 

-'-.113  .00-1023  LI=JJ1,KK1 

LLL=LL*  1 

DO  1022  1=1,3 
.„».„ DO  102  2  J=l,3 

00  1022  K^i,3        

1022  8S^TT.iG^CI' J>"WCI'K>-  TPCK'J'*-Lt-  J 

.............  00  1C23  1  =  1,3 

DO  102  3  J=t,3, 
W( I, J)=Q(I, J) 
t^„,   0(I#J)-0.0 

1023  CONTINUE 

j ......'  DO  1024  1-1,3  -        

00  1024  J=l,3   "  " " " 

1024  CAS^)!5uEJ)fTPUI'J'JJMHtJ'I) 

.:..._  ...  DO  1025  1  =  1,3 

TWK5>  =  TWK5fA(I)*TP(I»l,KKV     

1025  CONTINUE  U  M'RM 

WEEKSC  RANGE)  =  TWKS 

£,.""    TOTAL  THE  EMPLOYMENT  PROBAF IL I T T FS . 

fC     ^Dr^4^LY  BY  IHENO.  OF  MONTHS 
..£  .....  ™  f.HE  PROGRAM  Al  4.3  WEEKS  PER  MONTH 

100    DO  1026  1=1,4      -"■■"■'" -       —•—■■ - 

,nor   WKS=WKS*WEEKSCI> 

1026  CONTINUE 

-T.-1  ZZ.   WKS  =  WKS*4.  3*FACTC!P         ■  ■ -  -••■  - 

EKKSC JJ,KK)=WKS  - - 

123  CONTINUE 

C 
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IFC  I  I.  E Q.  2 > WRITE (6, 90  1?) 

IF( II. EQ. 3>WRITE(6,?013) 

DO    4142     1=2,22 
..  ..  IJJ.    DO    4  14?    J=?,?2  

,,,->       ^(J-l-T.I)EWKS(I,J)  =  0.0 
4142       CONTINUE 

0  0    4  14  1     1=?,?? 
rr  Uiil,JM£  ;  ?009  H  E  1VK  f,  (  I  ,  J  )  ,  j=2  ,2?  ) 
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unemployed  nonwhite  woman  aged  20-24  who  entered  this  particular  job-creation 
program  in  its  third  month  and  left  in  month  ten  would,  had  she  never 
encountered  Title  X,  have  experienced  2.81  weeks  of  employment  somewhere  in 
the  economy,  anyway. 

What  does  this  methodology  assume?   It  assumes  that  month-to-month  and 
season-to-season  changes  (not  levels)  in  the  employment  status-demography- 
labor-market  nexus  during  the  Title  X  period  are  likely  to  have  about  the 
same  shape  as  those  in  the  Urban  Institute  sample  period  1967-73.   And  it 
assumes  that  Title  X  is  generally  too  small  a  program  to  "feed  back"  and 
actually  affect  the  national  levels  of  the  instrumental  variables  (unemploy- 
ment and  vacancy  rates).   These  seem  to  us  to  be  reasonable  assumptions. 

The  assumption  that  last  month's  status  determines  this  month's  expec- 
tations —  within  a  cohort  —  is,  however,  certainly  not  appropriate  for 
all  enrollees  in  a  program  like  Title  X.   How,  for  example,  does  one  treat 
long  term  or  structurally  unemployed  persons  when  we  are  trying  to  estimate 
the  expected  number  of  weeks  of  employment  in  the  absence  of  Title  X?  We 
should  think  that  a  person  who  has  been  unemployed  for  26  weeks  will  be  less 
likely  to  have  the  same  employment  experience  (over  the  Title  X  period)  as 
a  person  who  was  unemployed  for  only  5  weeks  prior  to  Title  X.   However, 
the  Smith-Vanski-Holt  model  presented  previously  has  as  its  starting  point 
the  labor  force  status  of  the  person  in  the  month  prior  to  entrance  into 
the  model.   None  of  the  movement  in  the  model  depends  on  the  past  history 

of  the  individual  (before  month  t-1).   Duration  of  previous  employment  status 

.,    .  17/ 
is  not  considered. — 


—  For  the  US  population  sampled  randomly,  the  assumption  that  "history 
doesn't  count"  may  not  matter;  the  long  term  unemployed  are  a  small  enough 
proportion  of  the  total  labor  force  for  the  Urban  Institute  model  to  have 
unbiased  coefficients.   For  our  Title  X  sample,  on  the  other  hand,  the 
group  of  workers  who  were  previously  unemployed  for  26  weeks  or  longer  are 
in  fact  the  modal  group. 
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If  all  Title  X  participants  were  entered  into  the  model  without  regard 
to  their  employment  histories,  we  believe  we  would  seriously  overestimate 
the  number  of  weeks  a  person  would  have  been  employed  in  the  absence  of  the 
program.   We  have  therefore  adjusted  "entrance"  into  the  model  in  the 
following  manner: 

(1)  Some  persons  were  employed  at  the  time  of  their  entry  into  Title  X. 

Others  were  unemployed  or  had  not  been  in  the  labor  force  for  a  period 

of  26  weeks  or  less  prior  to  their  entering  Title  X.   Such  persons  are 

entered  into  the  model  according  to  their  status  in  the  month  prior  to 
Title  X  hiring. 

(2)  All  persons  who  were  unemployed  or  out  of  the  labor  force  for  more  than 

26  weeks  are  not  entered  into  the  model  at  all.   The  result  of  this 
qualification  is  to  count  all  of  the  Title  X  employment  of  these  persons 
as  an  addition  to  their  work  experience. 

This  decision  is,  of  course,  arbitrary.   The  26  week  cutoff  point  is  a 
simplifying  assumption,  which  seems  consistent  with  BLS  definitions  of  "long 

term  unemployment".   It  is  a  conservative  decision  rule,  since  81.7%  of  the  unem- 

18/ 
ployed  during  1976  had    been  in  that  state  for  26  weeks  or  less. —   And 

it  fits  well  with  the  intent  of  the  Title  X  program.   Since  Title  X  is  a 
countercyclical  program  and  targeted  at  those  who  were  unemployed  due  to 
the  economic  slump  dating  to  mid-1974,  the  degree  to  which  the  structural 
or  long  term  unemployed  were  aided  by  the  program  can  be  viewed  as  a  desirable 
side  benefit.   The  data  collected  by  Abt  for  EDA  show,  somewhat  surprisingly,  that 
approximately  one-third  of  those  working  on  Title  X  projects  had  been  unem- 
ployed for  over  26  weeks.   In  fact,  over  15  percent  of  the  participants  had 


18/ 

—  Bureau  of  Labor  Statistics,  Employment  and  Earnings,  Jan.  1977,  p.  149. 
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been  unemployed  for  more  than  52  weeks,  clearly  in  the  "structural"  category. 
These  figures  indicate  that  the  program,  even  though  not  required  to  do  so 
by  legislation  or  regulations,  employed  many  who  were  most  in  need  of  work. 
Finally,  we  should  note  that  the  Smith  et  al.  model  has  been  estimated 
with  national  CPS  and  Conference  Board  data.   Yet  we  are  using  it  to  simulate 
the  behavior  of  "control  groups"  of  workers  from  EDA  Title  X  target  areas. 
These  were  areas  of  high  unemployment  and  low  growth,  where  the  opportunities 
for  non-subsidized  employment  are  presumably  fewer  than  in  the  economy  as  a 
whole.   Thus,  our  estimates  of  alternative  work  opportunity  in  the  absence 
of  Title  X  must  be  biased  upward.   This,  then,  confers  a  further  conservative 
direction  to  our  estimates  of  the  net  job-creation  impact  of  Title  X. 


Comparing  the  Net  and  Gross  Private  Employment 
Benefits  from  the  Job  Opportunities  Program 


The  2,000  Title  X  workers  in  our  sample  were  employed  in  the  program 
for  an  average  of  35.4  weeks.   In  the  absence  of  the  Program,  we  estimate 
that  they  would  have  found  an  average  of  10.4  weeks  of  work  anyway  (using 
the  methodology  illustrated  above).   This  particular  federal  job-creation 
program  appears,  therefore,  to  have  created  an  average  of  25  person-weeks 
of  employment  for  each  enrollee  during  the  21  months  of  the  program's 
existence.   In  other  words,  only  about  71%  of  the  "new"  work  can  actually 
be  attributed  to  Title  X  itself.   Applying  this  fraction  to  the  total  number 
of  direct  full-time-equivalent  jobs  actually  filled  in  state  and  local 
development  projects,  which  we  estimate  at  76,900  on  the  assumptions  of  zero 
budget  substitution  and  displacement,  our  conclusion  is  that  the  Economic 
Development  Administration's  expenditure  of  $758  million  directly  produced 
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about  54,600  new  jobs  (not  counting  multiplier  effects).   To  the  extent 

that  Title  X  did  suffer  from  the  leakages  associated  with  budget  substitution 

19/ 
or  displacement,  the  net  direct  benefits  will  have  been  smaller. — 

In  this  paper,  we  have  reported  on  our  experiments  with  a  new  methodology 
for  program  evaluation.   In  the  absence  of  a  closely-matched  control  group 
of  non-Title  X  persons,  we  resorted  to  the  construction  of  what  amounts  to 
a  simulated  control  group  instead.   We  were  able  to  use  the  Smith-Holt-Vanski 
monthly  national  labor  market  transition  model  to  generate  a  set  of  expected 
month-to-month  careers  for  sixteen  age/race/sex  cohorts.   The  program- 
generated  work  experiences  of  a  sample  of  actual  Job  Opportunities  enrollees 
were  then  compared  systematically  with  the  simulated  careers  for  the  corre- 
sponding cohorts,  in  order  to  estimate  how  much  work  each  actual  enrollee 
might  have  been  expected  to  have  had,  given  his  or  her  age,  race  and  sex, 
even  in  the  absence  of  Title  X.   Had  we  tried  to  evaluate  the  direct  job- 
creation  impact  of  the  program  by  counting  the  actual  number  of  weeks  of 
work  funded  by  the  federal  government,  we  would  have  exaggerated  the  impact 
of  Title  X  (and,  it  should  be  noted,  its  social  cost  as  well)  by  about  30%. 

This  magnitude  seems  large  enough  to  warrant  serious  official  attention 
on  the  part  of  federal  agencies  like  EDA,  the  Dept.  of  Labor,  HUD,  etc. 
Consideration  should  be  given  to  the  "in-house"  development  of  detailed, 
standardized  stock-flow  models  such  as  the  one  employed  in  this  paper,  which 
could  be  used  routinely  to  simulate  control  group  behavior  in  the  process 
of  conducting  agency  program  evaluations.   Such  developments  need  not  and 


19/ 

The  income,  earnings  and  employment  multiplier  effects  of  this  EDA 

expenditure  are  estimated  in  Effects  of  Job  Creation,  Vol.  2,  Chap.  5. 

There,  an  additional  assumption  about   leakage  from  the  flow  of  funds 

is  introduced:   the  possibility  that  the  federal  government  will  fail  to 

respend  all  of  the  extra  tax  income  or  welfare  and  unemployment  insurance 

savings  that  are  generated  by  the  extra  jobs  created  by  the  program. 
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certainly  should  not  replace  experiments  with  true  control  groups.   But  the 
realities  of  survey  costs  and  time  pressures  point  to  the  need  for  computer- 
assisted  approximations.   Of  course,  only  with  much  further  research  and 
experimentation  will  it  be  possible  to  find  out  how  closely  the  behavior  of 
such  simulated  control  groups  corresponds  to  that  of  the  "real  thing",  and 
what  elements  of  model  design  are  the  most  instrumental  in  narrowing  the 
differences. 
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